System and method of polarization of an organic object with automated image analysis and organizatio

专利摘要:

公开号:NL2007152A
申请号:NL2007152
申请日:2011-07-20
公开日:2012-01-24
发明作者:Kenneth Dewane Owens Jr
申请人:Cognisense Labs Inc；
IPC主号:

专利说明:

SYSTEM AND METHOD OF POLARIZATION OF AN ORGANIC OBJECT WITH AUTOMATED IMAGE ANALYSIS AND ORGANIZATION
[0001] This application claims priority from • U.S. Application No. 12840334 entitled ‘AUTOMATED IMAGE ANALYSIS OF AN ORGANIC POLARIZED OBJECT’ filed on July 21, 2010 • U.S. Application No. 12840315 entitled ‘AUTOMATED ORGANIC POLARIZED OBJECT ORGANIZATION’ filed on July 21,2010 and; • U.S. Application No. 12841191 entitled ‘AUTOMATED POSITIONING OF AN ORGANIC POLARIZED OBJECT’ filed on July 22, 2010.
FIELD OF TECHNOLOGY
[0002] This disclosure relates generally to imaging and/or robotics and, in particular to system and method of polarization of an organic object with automated image analysis and organization.
BACKGROUND
[0003] Robotic systems may be used in an agricultural industry. For example, the robots may be used for ploughing purposes, cutting of standing crops, eliminating weeds, etc. The robotic system may also be used for planting. However, the planting of seeds and / or bulbs (organic polarized objects) may be a difficult task for robots, as the robots may crush the seeds, mishandle the seeds, or orient the seeds in the wrong orientation during planting. In addition, there may be labor issues associated with workers and automation would alleviate some of the difficulties for the agricultural industry.
SUMMARY
[0004] A system and method of polarization of an organic object with automated image analysis and organization is disclosed. In one aspect a method includes capturing an image of a first organic polarized object using an image capture device. In addition, the method includes collecting a first image data of the first organic polarized object. The method further includes algorithmically calculating a first dimension data of the first organic polarized object including a center and edges of the first organic polarized object using a processor. In addition, the method includes generating a first data table of the first dimension data of the first organic polarized object. The method also includes forming a training data set using transformation and /or scaling of the first data table of the first organic polarized object. The transformation may include, skewing, rotating, and the like.
[0005] The method may include capturing a second image or images of a second organic polarized object using the image capture device. In addition, the method may include collecting a second image data of the second organic polarized object. The method may further include algorithmically calculating a second dimension data of the second organic polarized object including, but not limited to the center and edges of the second organic polarized object using the processor. The dimension data described herein may be made of dimension parameters such as a width, a depth, a length, a distance, density, a curvature, a surface area, a volume, a narrow field, a broad field, edge, center and/or an angle. The method may also include calculating a high vote count using the second dimension data of the captured image of the second organic polarized object. The method may include creating a second data table of the second dimension data of the second organic polarized object. The method may include comparing the training data set to the second dimension data to identify the second organic polarized object and/or a precise orientation, location and size data of the second organic polarized object using the processor. The method may also include determining the dimension data as a distinct data for the second organic polarized object even if the second organic polarized object is adjacent, bordering, overlapping, underneath another object and/or an up-side down state. In addition, the method may include selecting a robotic end effector (e.g., robot arm/wrist) movement having an “n” degree of freedom of movement. The method may also include picking up the second organic polarized object using the robotic end effector in a precise orientation and location. The method may further include using the robotic end effector for transporting the second organic polarized object from a first location and first orientation to a predetermined second location and orientation in a specific tray with a slot.
[0006] In addition, the method may include having the slot in a specific shape to receive a first end of the second organic polarized object into the slot before a second end of the second organic polarized object, such that the first end of the second organic polarized object is oriented towards a narrow base of the slot and the second end is oriented towards a broad opening of the slot. The dimension data described herein may be made of dimension parameters such as a width, a depth, a length, a distance, density, a curvature, a surface area, a volume, a narrow field, a broad field, edge, center and/or an angle. In addition, the method may include finding the edges of the first organic polarized object from the captured image to generate the training set data. The method may also include correcting an angle of deposit based on the type of the slot. The method may further include depositing the organic polarized object in the precise orientation to the slot in the tray. In addition, the method may include permitting the “n” degrees of freedom of movement that include two or more of a moving up and down in heaving, a moving left and right in swaying, a moving forward and backward in surging, a tilting forward and backward in pitching, a turning left and right in yawing, a full axis motion in 360 degree rotation, a tilting side to side in rolling, and a moving along one or more of x, y, and z coordinate axes.
[0007] In another aspect, an organic polarized object detector system includes an image module to process an image of a first organic polarized object using a processor. In addition, the organic polarized object detector system includes an algorithm module to calculate a dimension data from a captured image of a first and a second organic polarized object. The organic polarized object detector system also includes a training set module to store a training set data. The organic polarized object detector system further includes a calibration module to align the coordinate systems of an image capture device and a robotic end effector. In addition, the organic polarized object detector system includes a movement module to direct the robotic end effector to perform a specific movement based on “n” degrees of freedom.
[0008] The system may include a transport module to determine the “n” degree of freedom movement for the robotic end effector. The system may also include the movement module to allocate a next best position for the second organic polarized object on a tray. In addition, the system may include a quality assurance module to guarantee the selection accuracy of the second organic polarized object through selection of a specific size, characteristics and /or shape of the second organic polarized object based on the training set data of first organic polarized object. The system may further include an alert module to indicate that a maximum threshold for depositing the second organic polarized object has been reached. The system may also include a change module to indicate a change of tray is warranted once the tray has reached a maximum capacity to hold the organic polarized object.
[0009] In yet another aspect, an organic polarized object detector apparatus includes an image capture device to record an image of a first organic polarized object and/or a second organic polarized object. In addition, the apparatus includes a data storage device to store a data set from the image capture device after a capture of the image. The apparatus also includes a processor to calculate the “n” degree of freedom movement for a robotic end effector using a training data set. The apparatus also includes a tray with two or more slots to hold the organic polarized objects at a specific coordinate. The apparatus further includes a signal device to indicate that the tray has reached a maximum capacity and to prompt a change for another empty tray.
[0010] In addition, the apparatus may include the robotic end effector controlled by one or more pneumatic cylinders. The pneumatic cylinder may include a first elongated extension and a second elongated extension to hold the second organic polarized object. The apparatus may also include the first elongated extension and the second elongated extension having a sensor device to detect the presence of an organic polarized object. The system may further include software to control the robotic end effector and ’n”’degree of freedom movement for the robotic end effector. The sensor device described herein may include, but is not limited, one of a capacitive sensor, a resistive sensor and/or an inductive sensor. Also, the image capture device described herein may include, for example, an infra-red device, a laser device, a camera, a biosensor, a color sensor, a heat sensor and/or a water sensor. The tray described herein may be a part of a large automated system.
[0011] In one aspect, the method includes spreading an organic polarized object in a single file on the conveyor belt when the organic polarized object is transported from organic polarized object storage. The method also includes updating an inventory control system through a wireless communication system when a count of the organic polarized object is communicated. The method further includes automatically filling the organic polarized object in a slot of a tray through a robotic end effector until a maximum fill capacity of the tray is achieved, wherein the slot is of a specific shape to receive a first end of the organic polarized object in the slot before a second end of the organic polarized object, such that the first end of the organic polarized object is oriented towards a narrow base of the slot and the second end is oriented towards a broad opening of the slot. The method furthermore includes covering the tray with a nutrient mixture, covering the tray with a basket, rotating the basket so that the organic polarized objects are correctly oriented for growing and removing the basket.
[0012] In another aspect, the apparatus includes an organic polarized object sorter machine to sort the organic polarized object in a single file on a conveyor belt. The apparatus also includes the conveyor belt to transport the organic polarized object to the robotic end effector which may place the organic polarized objects into a slot in a tray. The apparatus further includes an image capture device to record an image of one or more of a first organic polarized object and a second organic polarized object. The apparatus furthermore includes a data storage device to store a data set from the image capture device after the recording of the image. The apparatus furthermore includes a processor to calculate “n” degrees of freedom movement for the robotic end effector using a training data set. The apparatus furthermore includes a tray with a plurality of slots to hold the second organic polarized object at a specific slot. The apparatus furthermore includes a signal device to indicate that the tray has reached a maximum capacity and to prompt a change for another empty tray. The method further includes an apparatus for changing the tray orientation so that the organized polarized objects are in the correct orientation for growing.
[0013] In one or more embodiments, a hydroponic technology may be used to grow the organic polarized objects. As used herein the term hydroponics refers to a method of growing plants without soil. In these embodiments one or more essential nutrients may be introduced to the organic polarized objects through fluids (e.g., water) in the place of soil. Also, in these embodiments, the tray may contain an array of sharp pins and the robotic end effector may stick the organic polarized objects to the sharp pins. The trays may then be positioned in a liquid (e.g., water) to allow the organic polarized objects to grow when submerged in the liquid.
[0014] In one aspect, the method includes capturing an image of an organic polarized object through an image capture device. The method also includes, through the use of a processor, converting the captured image of the organic polarized object to an image data of the organic polarized object. The method further includes determining, through a processor, a first location, a first size and/ or a first orientation of the organic polarized object based on the image data of the organic polarized object. The method furthermore includes guiding through the processor, a movement of a robotic arm towards the organic polarized object, along one or more of x, y, and z coordinate axes. The method furthermore includes applying a pressure to secure the organic polarized object at the first location with the first orientation.
[0015] The method furthermore includes securing, through a robotic arm the organic polarized object to move the organic polarized object to a predetermined location. The method furthermore includes adjusting the secured organic polarized object from the first orientation to a predetermined orientation. The method furthermore includes positioning the organic polarized object of the predetermined orientation in a predetermined location. The method furthermore includes automating, through a training data set, the positioning of the organic polarized object of the predetermined orientation at the predetermined location. In some embodiments, the predetermined location and/ or the predetermined orientation are selected by a user In another aspect, the system includes an image capture device to capture an image of an organic polarized object. The system also includes a processor to determine a first location and/or a first orientation of the organic polarized object. The system further includes a robotic hand to secure the organic polarized object. The system furthermore includes a robotic arm to adjust the first location and/ or the first orientation of the organic polarized object to a predetermined location and/ or a predetermined orientation.
[0016] In yet another aspect, the apparatus includes a robotic arm to adjust a first location and/or a first orientation of an organic polarized object to a predetermined location and/ or a predetermined orientation. The apparatus also includes a robotic hand to secure the organic polarized object. The apparatus further includes a first pneumatic cylinder to reduce damage to the organic polarized object by adjusting a pressure applied to the organic polarized object when the organic polarized object is secured through the robotic hand. The apparatus furthermore includes a valve to reduce damage to the organic polarized object by adjusting the pressure of a compressed air of the pneumatic cylinder when the organic polarized object is secured through the robotic hand.
[0017] The methods, systems, and apparatuses disclosed herein may be implemented in any means for achieving various embodiments, and may be executed in a form of a machine-readable medium embodying a set of instructions that, when executed by a machine, cause the machine to perform any of the operations disclosed herein. Other features will be apparent from the accompanying drawings and from the detailed description that follows.
BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Example embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements and in which:
[0019] Figures 1A-1B illustrate an apparatus for automated organic polarized object organization, in accordance with one or more embodiments.
[0020] Figure 2 illustrates a first stage during a process of automated organic polarized object organization, in accordance with one or more embodiments.
[0021] Figure 3A-3B illustrates a second stage during the process of automated organic polarized object organization, in accordance with one or more embodiments.
[0022] Figures 4A-4B illustrate a third stage during the process of automated organic polarized object organization, in accordance with one or more embodiments.
[0023] Figure 5 illustrates selection of one or more organic polarized objects for automated organization of the organic polarized objects, in accordance with one or more embodiments.
[0024] Figure 6 is a block diagram illustrating an organic polarized object processing system, in accordance with one or more embodiments.
[0025] Figure 7 is a diagrammatic process flow illustrating a process of object detection and placement, according to one or more embodiments.
[0026] Figure 8 is a perspective view of a part of a robotic end effector, according to one or more embodiments.
[0027] Figure 9A is a diagrammatic process flow illustrating generation of a training data set, according to one or more embodiments.
[0028] Figure 9B is a diagrammatic process flow illustrating step A and step B of Figure 1A, according to one or more embodiments.
[0029] Figure 10A is a schematic view illustrating generation of a training data set, according to one or more embodiments.
[0030] Figure 11B is a flow chart illustrating a process of comparing a dimension data of an organic polarized object of interest to a training data set to identify if the organic polarized object of interest possesses the desirable shape and size, as specified in the training data set, according to one or more embodiments.
[0031] Figure 12 illustrates a right side view of an apparatus for positioning an organic polarized object at a predetermined location with a predetermined orientation, according to one or more embodiments.
[0032] Figures 13-15 illustrate positioning an organic polarized object in a predetermined location at a predetermined orientation, according to one or more embodiments.
[0033] Figure 16 is a continuation of the process of Figure 3B, illustrating additional operations, according to one or more embodiments.
[0034] Figure 17 is a schematic view illustrating a system of object detection and placement, according to one or more embodiments.
[0035] Figure 18 is a block diagram of a processing system of organic polarized object detection and/ or positioning, according to one or more embodiments.
[0036] Figure 19A is a process flow illustrating identification of organic polarized objects and placement of the same using the automated system, according to one or more embodiments.
[0037] Figure 19B is a continuation of process flow of Figure 19A, illustrating additional operations, according to one or more embodiments.
[0038] Figure 19C is a continuation of process flow of Figure 19B, illustrating additional operations, according to one embodiment.
[0039] Figure 20 is a process flow illustrating generating an orientation, size and the three-dimensional coordinates of the organic object of interest and communication of the same to the robotic end effector.
[0040] Figure 21 is a process flow diagram of determining a center of the contour of the binary image, according to one or more embodiments.
[0041] Figure 22 is a process flow illustrating a method of automated organic polarized object organization, in accordance with one or more embodiments.
[0042] Figures 23A-23B show a process flow illustrating a vision based method of automated organic polarized object organization, in accordance with one or more embodiments.
[0043] Figure 24 is a process flow diagram illustrating a method of positioning an organic polarized object at a predetermined location with a predetermined orientation, in accordance with one or more embodiments.
[0044] Other features of the present embodiments will be apparent from accompanying Drawings and from the Detailed Description that follows.
DETAILED DESCRIPTION
[0045] A system and method of polarization of an organic object with automated image analysis and organization is disclosed. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments. It may be evident, however, to one skilled in the art that the various embodiments may be practiced without these specific details.
[0046] Figures 1A-1B illustrate an apparatus 100 for automated organic polarized object organization in accordance with one or more embodiments. For purposes of illustration, the detailed description refers to an organic polarized object; however the scope of the method, the system, and /or the apparatus 100 disclosed herein is not limited to a single organic polarized object but may be extended to include an almost unlimited number of organic polarized objects. As used herein, the term “organic polarized object” refers to any organic object of a regular or an irregular shape and form, having a proximal end and a distal end, and hence an orientation. Example of the organic polarized object 102 may include, but is not limited to, a seed, a plant bulb, a resting stage of a seed plant, a sapling, and the like.
[0047] The apparatus 100 includes an organic polarized object sorter machine to sort the organic polarized object 102 in a single file on a conveyor belt 160. The conveyor belt 160 may be used to transport one or more organic polarized objects (e.g. organic polarized object 102) from the organic polarized object sorter machine towards a robotic end effector 134. The robotic end effector 134 may be configured to fetch the organic polarized objects from the conveyor belt 160 and place the organic polarized objects into one or more slots in a tray 148. In one or more embodiments, the movement of the robotic end effector 134 may be controlled by pneumatic cylinders (not shown). The pneumatic cylinders may be attached to one or more elongated extensions (e.g., a first elongated extension 136A and a second elongated extension 136B) to hold the organic polarized objects (e.g. organic polarized object 102) to transfer the organic polarized objects into the slots and/or pins of the tray 148 at desired location and /or in a desired orientation. In one or more embodiments, the first elongated extension 136A and the second elongated extension 136B may include a sensor device (not shown) to sense the presence of the organic polarized objects. Example of the sensor device may include, but is not limited, one of a capacitive sensor, a optical sensor, a resistive sensor, an inductive sensor, and the like. In one or more embodiments, the sensor device may also relay / communicate a count of the organic polarized objects that are processed through the apparatus 100, to a data processing system (e.g. computer 138). The communication may be through a wired and /or a wireless communication. In one or more embodiments, an application may control the robotic end effector 134 and movement of the robotic end effector 134.
[0048] In one or more embodiments, the apparatus 100 also includes an image capture device 120, a data storage device, a processor, the tray 148, and a signal device. The image capture device 120 may be configured to capture an image of an organic polarized object. For example, a first image of a first organic polarized object and/or a second image of a second organic polarized object may be captured through the image capture device 120. The image capture device 120 may record the image after capturing. Example of the image capture device 120 includes, but is not limited to, a digital camera, a video camera, a probe, an optical device, an infra-red device, a biosensor, a color sensor, a heat sensor, a water sensor, and a laser device. In one or more embodiments, the captured image may be transferred to a data storage device. The data storage device may store a data set from the image capture device 120 after the recording of the image. The processor may calculate “n” degrees of freedom movement for the robotic end effector 134 using a training data set. The “n” degrees of freedom of movement may include, but is not limited to a moving up and down in heaving, a moving left and right in swaying, a moving forward and backward in surging, a tilting forward and backward in pitching, a turning left and right in yawing, a full axis motion in 360 degree rotation, a tilting side to side in rolling, and a movement along one or more of x, y, and z coordinate axes. Further, the processor may be configured to algorithmically calculate a dimension data from the captured images (e.g., the first image and the second image). Examples of the dimension data may include, but is not limited to, one or more of a width, a depth, a length, a distance, intensity, a curvature, a surface area, a volume, a narrow field, a broad field, edges, center, and an angle.
[0049] The tray 148 may include one or more slots and /or pins to hold the organic polarized objects (e.g. the first organic polarized object 102) at specific slots/pins. In one or more embodiments, the signal device may be configured to indicate that the tray 148 has reached a maximum capacity and may prompt a change for another empty tray. In one or more embodiments the tray 148 may be a part of a large automated assembly system. In one or more embodiments, the tray 148 may be made of one or more of a biodegradable material, a plastic material, a reusable material, and /or an array of sharp pins for holding the organic polarized objects.
[0050] Also illustrated in Figure 1A is a nutrient mixture filling station 151. The nutrient mixture filing station 151 may fill a nutrient mixture 162 into the tray 148 filled with the organic polarized objects. The nutrient mixture 162 may be stored in a nutrient mixer 155. In one or more embodiments, a fill station sensor 153 coupled to the nutrient filling station 151 may communicate one or more of a level of nutrient mixture 162 in the station, change of the tray 148, and a count for an inventory control system to the data processing system (e.g. computer 138) through a wireless communication 108 or a wired communication. The inventory control system may keep track of a number of organic polarized objects used, a number of trays filled, a number of organic polarized objects not selected, a level of the nutrient mixture 162 and a number of organic polarized objects left in the organic polarized object storage 104. The wireless communication system 108 may be designed to include one or more of a Bluetooth, a Zigbee, a WiFi, a WiMax, a PoE, and a Wibree interface. Figure 1B illustrates a basket flipping station 152 of apparatus 100 of Figure 1A, in accordance with one or more embodiments. In one or more embodiments, as illustrated in Figure 1B, the tray 148 filled with the organic polarized objects and the nutrient mixture 162 may be a covered with a basket 150 at the basket flipping station 152. The basket 150 may then be flipped over at the basket flipping station 152 through a basket flipping device 156. The tray 148 may then be removed from the flipped basket 150 containing the organic polarized objects covered with the nutrient mixture 162 as illustrated in Figure 1B. In one or more embodiments, the tray 148 may be removed through the basket flipping device 156. In one or more embodiments, after removing the tray 148, additional nutrient mixture 162 may be added to cover the organic polarized objects.
[0051] In one or more embodiments, personnel may be updated about a status of an inventory item. Examples of the inventory item may include, but is not limited to the organic polarized object 102, the nutrient mixture 162, the tray 148 and the basket 150. Updating the personnel may be conducted via one or more communication techniques including, for example, a cell phone, a PDA and a computer.
[0052] Figure 2 illustrates a first stage during a process of automated organic polarized object organization, in accordance with one or more embodiments. During the first stage, one or more organic polarized objects may be transferred from the organic polarized object storage 104 to an organic polarized object sorter 106 through an upswing conveyor belt 124. Each organic polarized object 102 may be positioned on a supporting structure 202 on the upswing conveyor belt 124. The arrival of each of the organic polarized objects at the organic polarized object sorter 106 may be sensed through a sensor 208 and may be communicated to a data processing system (e.g. computer 138). The communication may be through a wireless communication 108 and / or a wired communication. The data processing system may count the number of organic polarized objects arriving at the organic polarized object sorter 106. In one or more embodiments the data processing system may update a database based on the counting. In one or more embodiments the data processing system may communicate the number of the organic polarized objects to a client device 142 through a network 140. The network 140 may include, but is not limited to, a local area network, a wide area network, a wired network, a wireless network, a mobile communication network, and the like. Examples of the client device 142 includes, but is not limited to a portable computing device, a laptop, a desktop, a mobile communication device, a personal digital assistant, and an electronic communication device.
[0053] Figure 3A-3B illustrates a second stage during the process of automated organic polarized object organization, in accordance with one or more embodiments. During the second stage, one or more organic polarized objects can be sorted through a robotic end effector 134 for organizing in one or more slots and / or pins of a tray 148. The robotic end effector 134 may be trained to automatically sort the organic polarized objects using a training data set. In one or more embodiments, the training data set may include information (e.g., coordinate information, dimension information) that can be used for training and executing certain functionalities through systems such as a robotic vision system. In an example embodiment, the training data set described herein may be used for generating commands or providing machine instructions for the robotic end effector 134 to perform one or more tasks. In one or more embodiments, an image of the organic polarized object 102 may be captured using the image capture device 120. The image capture device 120 may be coupled to the robotic end effector 134. In one or more embodiments the captured image may be transmitted to the data processing system (e.g. computer 138) for further processing. Alternatively, in one or more embodiments further processing may be performed within the robotic end effector 134.
[0054] The captured image may be processed to obtain an image data. The image data may include, but is not limited to, a width, a depth, a length, a distance, intensity, a curvature, a surface area, a volume, a narrow field, a broad field, edges, center, an angle, and the like. The image data may be used to create a data set for the organic polarized object 102. The data set may be compared with the training data set to identify the organic polarized objects (e.g. the organic polarized object 102) and /or a precise location, orientation, shape, and/or size data of the organic polarized objects. If the comparison yields a positive result indicating that the location, orientation and/or size of the organic polarized objects matches one or more specifications including a specific shape, a specific size, specific location, and /or specific orientation, then the organic polarized object 102 may be selected and fetched from the conveyor belt 160 using the robot end effector 134. In one or more embodiments, if the comparison yields a negative result, then the organic polarized object 102 may be recycled for future processing. In one or more embodiments, on selection, the organic polarized object 102 may be secured between a first elongated extension 136A and a second elongated extension 136B of the robotic end effector 134 to grasp the organic polarized object 102. After securing, the organic polarized object 102 may be positioned in one of the slots and /or pins of the tray 148.
[0055] In one or more embodiments, a data regarding the tray 148 being filled, a number of available slots in the tray 148 being filled, and/ or a number of trays filled may be sensed through the image capture device 120 and the data may be communicated to the data processing system through the wireless /wired communication system. When the tray 148 may be completely filed, a filled tray 346 may be obtained and the filled tray 346 may be covered with a nutrient mixture 162 at the nutrient filling station 151. In one or more embodiments, the filled tray 346 may be partially covered with the nutrient mixture 162 at the nutrient filing station 151. In one or more embodiments, the nutrient mixture 162 may be transported through a duct from the nutrient mixer 155 operatively coupled to the nutrient filling station 151, to a nutrient bin 343. The filled tray 346 (filled with the organic polarized objects) to be filled with the nutrient mixture 162 may be positioned below the nutrient bin 343 through the conveyor belt 160. The nutrient mixture 162 may be dispensed into the filled tray 346 from the nutrient bin 343 to partially or fully fill the tray 148 with the nutrient mixture 162 as illustrated in Figure 3B.
[0056] Figures 4A-4B illustrate a third stage during the process of automated organic polarized object organization, in accordance with one or more embodiments. During the third stage, the filled tray 346 containing the organic polarized objects covered with the nutrient mixture 162 may be covered with the basket 150. One or more dimensions of the basket 150 may match one or more dimensions of the filled tray 346. In one or more embodiments, the filled basket 150 may be flipped at a basket flipping station 152 through a basket flipping device 156 so as to correctly orient the organic polarized objects for growing as illustrated in Figure 4A. The basket 150 containing the filled tray 346 with the organic polarized objects may then have the tray 346 removed through the basket flipping device 156 or any other device. In one or more embodiments, the basket 150 containing the organic polarized object 102 with the nutrient mixture 162 may be further filled with nutrient mixture 162 and stacked in a vertical form and/or a horizontal form on a platform to form a basket stack 422 as illustrated in Figure 4B. The basket stack 422 may be stored in a cold storage facility for a length of time. Then the basket stack 422 may be transported to a growing area. In one or more embodiments, the fill station sensor 153 senses change of the basket 150, and /or a count for an inventory control system and communicates the sensed change of the basket 150, and the sensed count for an inventory control system to the data processing system. A sensor 409 may sense the number of baskets stacked together to form the basket stack 422. In one or more embodiments, the nutrient mixture 162 may be added to the organic polarized objects after flipping the basket over and removing the tray 346. The flipped basket containing the organic polarized objects may be transported to the nutrient filing station 151 through the conveyor belt 160.
[0057] Figure 5 illustrates selection of one or more organic polarized objects for automated organization of the organic polarized objects, in accordance with one or more embodiments. As illustrated in Figure 5, at step A, a first image of a first organic polarized object 503 is captured through the image capture device 120. The captured first image is processed via a data processing system (e.g. computer 138) to determine a first image data. A first dimension data of the first organic polarized object 503 may be calculated based on the first image data. The first dimension data may be algorithmically calculated. The first dimension data includes, but is not limited to, one or more of a width, a depth, a length, a distance, intensity, a curvature, a surface area, a volume, a narrow field, a broad field, edges, center and an angle. In one or more embodiments, a data table of vectors may be generated from the first dimension data of organic polarized object 503. In one or more embodiments, a training data set may be formed by transforming (e.g., skewing, rotating, scaling) the data table of vectors of the first organic polarized object 503. The training data set may determine a desirable shape, size, location, and / or orientation of one or more organic polarized objects to be selected from among the organic polarized objects for positioning in the tray 148.
[0058] Further, at step B, a second image of a second organic polarized object 505 may be captured using the image capture device 120. A second image data of the second organic polarized object 505 may be computed using the captured second image. Further, a second dimension data may be computed based on the second image data through a processor coupled to the data processing system (e.g. computer 138). At step C, the training data set may be compared to the second dimension data through the processor to identify the shape, size, location and /or orientation of the second organic polarized object. On comparison, if the second organic polarized object 505 possesses the desirable shape, size, location and /or orientation, the processor may transmit a command to the robotic end effector 134 to select the second organic polarized object 505 to be positioned in the tray 148. In one or more embodiments, at step D, the robotic end effector 134 secures the second organic polarized object 505 between a first elongated extension 136A and a second elongated extension 136B and positions it in the tray 148. Similarly, the third organic polarized object 507 may be selected owing to suitability of shape, size and may be positioned in the tray 148 by the robotic end effector 134. Further, if the shape, size of one or more organic polarized objects (e.g. third organic polarized object 507) is not desirable, the organic polarized objects may be rejected or recycled for further processing.
[0059] Figure 6 is a block diagram illustrating an organic polarized object processing system 600, in accordance with one or more embodiments. The organic polarized object processing system 600 includes an algorithm module 602, an organic object detector module 604, an image capture module 606, a count sensor module 608, a robotic end effector sensor module 610, a motion module 612, a quality assurance module 614, a nutrient filling station module 616, a change module 618, an alert module 620, a transport module 622, a sensor control module 624, a training module 626, a calibration module 628, and a basket flipping module 630. In one or more embodiments, the algorithm module 602 includes an algorithm to perform one or more functions including, but not limited to regulating one or more operations of the organic object detector module 604, the image capture module 606, the count sensor module 608, the robotic end effector sensor module 610, the motion module 612, the quality assurance module 614, the nutrient filling station module 616, the change module 618, the alert module 620, the transport module 622, the sensor control module 624, and the training module 626. In one or more embodiments, the organic object detector module 604 may detect an organic polarized object of a desired shape, size, location and/or orientation from among multiple organic polarized objects. The organic object detector module 604 may coordinate with the image capture module 606 to perform the detection. In one or more embodiments, the image capture module 606 operatively couples one or more image capture devices to the organic object detector module 604.
[0060] In one or more embodiments, image capture module 606 also may control the functioning of the image capture devices (e.g. image capture device 120). The count sensor module 608 may coordinate with a sensor coupled to the robotic end effector and maintain a count of the number of organic polarized objects arriving from organic polarized object storage and update a database every time a new organic polarized object arrives. In one or more embodiments, the robotic end effector sensor module 610 may sense the various positions and movement of the robotic end effector 134. The motion module 612 regulates the movement of the robotic end effector 134 to automate organizing of the organic polarized objects in one or more slots of the tray 148. The motion module 612 may also regulate the movement of one or more parts of the robotic end effector 134, including, for example, elbow joint, a first elongated extension 136A, and a second elongated extension 136B. The motion module 612 coordinates with the robotic end effector sensor module 610 to regulate the movement of the robotic end effector 134.
[0061] In one or more embodiments, a quality assurance module 614 maintains uniformity of the organic polarized objects in the slots of the tray 148 and also controls the functioning of various modules to maintain uniformity and quality. The uniformity may be in terms of one or more of a number of organic polarized objects per tray, the quality of the organic polarized objects in the tray, a permissible amount of size and shape variation from a predetermined size and shape of the organic polarized objects, and an order of organizing the organic polarized objects in the tray. The nutrient filling station module 616 may control one or more functionalities of a nutrient filling station 151 including, but not limited to, mixing nutrients in a required proportion, transferring the nutrient mixture 162 to a nutrient bin 343, controlling the pouring of the nutrient mixture 162 into the basket 150 containing the filled tray 346, adjusting the position of the basket below the nutrient bin 343, and the like. The change module 618 may monitor that the tray 148 has reached a maximum capacity and to prompt a change for another empty tray.
[0062] In one or more embodiments, the alert module 620 alerts the change module to prompt when the tray 148 reaches a maximum capacity. The transport module 622 may control transporting the trays, the baskets, and /or the organic polarized objects on the conveyor belt 160. In one or more embodiments, the calibration module 628 may align coordinate systems of an image capture device (e.g. image capture device 120) and /or the robotic end effector. The basket flipping module 630 may communicate with the transport module 622 to coordinate basket flipping. In one or more embodiments, the basket flipping module 630 may also be configured to control and/or coordinate removal of the tray 148 after flipping the basket 150 over. The sensor control module 624 may control one or more sensors (e.g. sensor 336) coupled to the robotic end effector 134 and the nutrient filling station 151 and coordinates with various modules. The training module 626 may create and maintain a training data set by coordinating with the image capture module 606.
[0001] Figure 7 is a diagrammatic process flow illustrating a process of object detection and placement, according to one or more embodiments. The automated system described herein may be used for various purposes such as farming, packing, etc. In one or more embodiments, the automated system described herein may be comprised of a data processing system 708 (e.g. a computer), an image capture devices 706, a robotic end effector 710 communicatively coupled to each other. The object described herein may be any object, preferably an organic polarized object to be placed from one location and orientation to another location and orientation in an organized manner. As used herein, the term “organic polarized object” refers to any organic object of a regular or an irregular shape and form, having a proximal end and a distal end, and hence an orientation. The organic polarized object includes, for example, a seed, a plant bulb, a resting stage of a seed plant, and a sapling.
[0002] In one embodiment, the organic polarized object may be detected using a cellular component such as Adenosine Tri-Phosphate (ATP) (e.g., ATP being a content of the live organic polarized object). In alternate embodiments, objects other than the organic polarized object may be detected using one or more appropriate techniques. In one or more embodiments, the automated system as described herein may be trained to sense an organic polarized object and place the detected object in a tray 712 provided thereof. Step A of the process illustrated in Figure 7 may be a training step. In step A, an image 702 of a detected object (e.g. an ideal organic polarized object 703A), may be captured using an image capture device 706 such as a digital camera, scanner, infra red camera device and laser camera device, and the like. The captured image 702 may be communicated to the data processing system 708. In one or more embodiments, the communication may be enabled through a wired communication (e.g., universal serial bus), and/or through a wireless communication (e.g. Bluetooth).
[0003] Examples of the data processing system 708 may include, but is not limited to, a computer, a microcontroller embedded device, and the like. The data processing system 708 may collect one or more captured images of the ideal organic polarized object 703A. The collected captured images may be stored in a storage device (e.g., accumulators in the memory) of the data processing system 708 provided thereof. Further, the collected captured images may be analyzed (e.g., using image processing) to detect and derive edges (e.g., through edge detection), center and other shape information. The edge detection may be a process of identifying points in the digital image at which the image brightness changes sharply or more formally has discontinuities. A series of curves that indicate boundaries may be obtained from the image. Furthermore, the center of the object 703A may be determined using appropriate methods using the detected edges. In addition, vectors may be generated from the captured image 702 of the ideal organic polarized object.
[0004] In addition, a table (e.g., data table) that includes the vector data, referred to herein after as the data table of vectors, may be generated. Furthermore, a training data set may be generated (e.g., based on ideal organic polarized object 703A) by transforming the data table of vectors. In one or more embodiments, the data table of vectors described herein may be converted into command instructions using software to enable the robotic end effector 710 to move to a specific location and /or orientation. Furthermore, the robotic end effector 710 may be configured by the data processing system 708 to pick up the organic polarized object and place it in a required location (e.g., a tray 712) and orientation. For example, the organic polarized objects (e.g. tulip bulbs) may be placed in slots of the tray 712 with an orientation such that the narrow end faces downwards and the broad end faces upwards. Alternatively, the tray 712 may include one or more pins at appropriate locations instead of slots so that the tulip bulbs can be fixed to the pins to hold the tulip bulbs for hydroponic growth technology. In one or more embodiments, hydroponic growth technology may be a technology that involves a method of growing plants using nutrient solutions, in water, without soil. The embodiments described herein may be used for supporting hydroponic technology as well.
[0005] In one or more embodiments, the tray 712 may be a grid of which coordinates, shape and size information may be programmed into the data processing system 708. In addition, the tray 712 may include one or more slots/pins to hold the organic polarized objects. In alternate embodiments, the grid information may be manually input by an operator of the automated system. The robotic end effector 710 may be controlled by pneumatic cylinder or any other suitable system provided thereof. In one example embodiment, the pistons in the pneumatic cylinder as described herein may be driven by compressed air.
[0006] Steps B-D of the process illustrated in Figure 7 represent a real-time operation. In step B, the images of other organic polarized objects (e.g., 707A, 711A, etc.) may be captured and processed in a similar method to generate image data of each of the organic polarized objects in the data processing system 708. In one or more embodiments, dimension data including, but not limited to, edges may be computed. In addition, edges of the organic polarized object may be computed using the image data of the organic polarized object. Furthermore, in one or more embodiments, a binary image that displays contours in a monochrome color may be generated. In one or more embodiments, the training data set may be used for casting votes for determining a center of the organic polarized object (e.g., 707A, 711 A, etc.). In one or more embodiments, the data table of vectors may be rotated “R” times, scaled “S” times and voting process may be performed. Furthermore, data collected from the voting may be stored until a vector set has been scaled “S” number of times and rotated “R” number of times.
[0007] The vote counts for each rotation and scaling may compare the training data set of the ideal organic object with the organic polarized object and may be stored in a vote compare table. A particular X, Y coordinate with the highest vote count among the other votes in the vote compare table may be located. Furthermore, the center of the contours of the binary image may be located at the particular X, Y coordinate if a vote count at the particular X, Y coordinate exceeds a threshold minimum vote count. Furthermore, in one or more embodiments, vector computation may be performed and training coordinates may be recorded and a training set data may be generated.
[0008] Further, the training data set of the ideal object may be used for comparing with the generated image data for selecting/discarding the organic polarized object, and for identifying a precise orientation data, size data, shape data and a location data of the organic polarized object (e.g., 707A, 711A, etc.). If the comparison evaluates to true, then the organic polarized object may be selected for placement. In one or more embodiments, the comparison evaluates to be true only if the dimension data of the organic polarized object of interest matches substantially with the data table of the training data set of the ideal organic polarized object 703A. In alternate embodiments, the organic polarized object may be rejected if the comparison evaluates to be false. In one or more embodiments, the size and orientation of the organic polarized object may be determined from the S, R value respectively associated with the vote compare table on which the particular X, Y coordinate is found if the vote count at the particular X,Y coordinate exceeds the threshold minimum vote count. Also, commands based on the generated data table may be communicated to the robotic end effector 710. Furthermore, in step C, of the process illustrated in Figure 7, based on the commands generated, the robotic end effector 710 may be configured to pick up the selected organic polarized objects and place/fix them in a designated location and a designated orientation such as a tray 712 provided thereof.
[0009] In one or more embodiments, the dimensions or the coordinates of the tray 712 may be input to the data processing system 708. In alternate embodiments, the tray 712 may be scanned by the image capture device 706 to determine dimensions or co-ordinates of the tray 712 to place the organic polarized objects. The process may continue until all or selected organic polarized objects of the set of the organic polarized objects are relocated. All the process described herein may be processed through a processor of the data processing system 708. The process described herein may be programmed using necessary programs written in any suitable language (e.g., C, java, etc.). In addition, the programs may be modified, new programs may be added and or programs may be deleted through interfaces provided thereof. The process of the automated system may be controlled through software. The term “software” described herein may include software, firmware, wired or programmed hardware, or any combination thereof as appropriate. Furthermore, image processing tools may also be used for processing the images of the organic polarized object. Also, an interface may be designed and implemented to enable communication between the robotic end effector 710 and the data processing system 708. Alternatively, the existing technology may be used for interfacing the data processing system 708 and the robotic end effector 710.
[0010] In addition, the commands to the robotic end effector 710 may be processed by a controller and other hardware of the robotic end effector 710. Furthermore, a pipelining process may be implemented to enable faster processing. For example, organic polarized objects in close proximity to the analyzed organic polarized object in the tray 712 may be analyzed during displacement of the organic polarized object to enable selection of the next organic polarized object. In one or more embodiments, the organic polarized objects to be analyzed may be adjacent, bordering, overlapping, an underneath or anywhere relative to a current organic polarized object of interest. In step D of the process illustrated in Figure 7, once the tray 712 is full, a new tray may be placed and the process would be started again from step B followed by step C till the tray is filled and then step D would be initiated.
[0011] Figure 8 is a perspective view of a portion of the robotic end effector 834, according to one or more embodiments. For purposes of illustration, the detailed description refers to an organic polarized object; however the scope of the method, the system, and the apparatus disclosed herein is not limited to a single organic polarized object but may be extended to include an almost unlimited number of organic polarized objects. As used herein, the term “organic polarized object” may refer to any organic polarized object of a regular or an irregular shape and form, having a proximal end and a distal end, and hence an orientation. Examples of the organic polarized object include, but are not limited to a seed, a plant bulb, a resting stage of a seed plant, and/ or a sapling.
[0012] The robotic end effector 834 includes a robotic hand 830, one or more pneumatic cylinders (e.g. a first pneumatic cylinder 814, a second pneumatic cylinder 816, and a third pneumatic cylinder (not shown)), one or more elongated extensions (e.g., a first elongated extension 836A, and a second elongated extension 836B), and a valve (not shown). In one or more embodiments the robotic end effector 834 includes a sensor 820 to sense various parameters associated with the organic polarized object. Examples of the parameters include, but are not limited to, one or more dimensions, a stress withstanding capacity, one or more contours on a surface of the organic polarized object, and the like. The data processing system 808 may be enabled to capture an image of an organic polarized object through the image capture device 706 to determine a first location and/ or a first orientation of the organic polarized object based on the image. In some embodiments, the image capture device 706 may be operatively coupled with the robotic end effector 834. In some other embodiments, the image capture device 806 may be external to the robotic end effector 834. In some embodiments, one or more feature extraction techniques including, but not limited to a generalized Hough transform may be used to determine the location, size, shape and orientation of the organic polarized object based on the image data.
[0013] In some embodiments, the first elongated extension 836A and/or the second elongated extension 836B, include a cuff. The cuff may be discshaped and/ or concave so as to secure the organic polarized object that is curved in shape and delicate in nature. In some embodiments, the cuff may be made of a flexible material to reduce the damage to the organic polarized object. In one or more embodiments, the elongated extensions may be made of the flexible material. Example of the flexible material includes, but is not limited to rubber and flexible plastic. In some embodiments, the first pneumatic cylinder 814 and the second pneumatic cylinder 816 regulate the movement of the first elongated extension 836A and the second elongated extension 836B respectively. In some embodiments, the third pneumatic cylinder 836C (not shown) regulates the movement of the robotic hand 830. For purposes of illustration, the detailed description refers to a first pneumatic cylinder, a second pneumatic cylinder, and a third pneumatic cylinder; however the scope of the method, the system, and /or the apparatus disclosed herein is not limited to the first pneumatic cylinder, the second pneumatic cylinder, and the third pneumatic cylinder but may be extended to include an almost unlimited number of pneumatic cylinders or other actuators.
[0014] Multiple pneumatic cylinders may be used to regulate the movement of the robotic end effector 834 and various components of the robotic end effector 834 therein. In one or more embodiments, motion of the robotic end effector 834 includes “n” degrees of freedom of movement. The “n” degrees of freedom of movement includes, but is not limited to, a moving up and down in heaving, a moving left and right in swaying, a moving forward and backward in surging, a tilting forward and backward in pitching, a turning left and right in yawing, a full axis motion in 360 degree rotation and a tilting side to side in rolling.
[0015] Figure 9A is a diagrammatic process flow illustrating generation of a training data set 920, according to one or more embodiments. Figure 9A, in particular may be used to briefly elaborate the step A of the Figure 7. At step 902, an image 902 of the organic polarized object 903A may be captured using the image capture device 106 (e.g., a digital camera). The captured image 902 of the organic polarized object 903A may be communicated to the data processing system 708 (e.g., a computer), to an image processor enabled system or to any appropriate system for analysis and processing. An image data may be determined based on the image 902. At step 904, edges of the organic polarized object 903A may be computed and mapped using the image data to determine the shape of the organic polarized object 903A. Furthermore, at step 906, a center 914 of the organic polarized object 903A may be identified based on the mapped edges of the organic polarized object 903A. In one or more embodiments, at step 908, the image data associated with the organic polarized object 903A may be analyzed and one or more vectors 912 may be computed based on the image data. In one or more embodiments, at step 910, one or more training coordinates may be recorded and a training data may be generated. In one or more embodiments, a table that includes the training data may be generated. In one or more embodiments, the table may be termed as the training data set 920.
[0016] In one or more embodiments, the training data set 920 may be information (e.g., coordinate information, dimension information) that can be used for training and executing certain functionalities through systems such as a robotic vision system. In an example embodiment, the training data set 920 described herein may be used for generating commands or providing machine instructions for the robotic end effector 834 to perform aforementioned task. Furthermore, the robotic end effector 834 may be aligned with respect to the organic polarized object 903A to pick up the organic polarized object 903A and to place it in a specific location with a specified orientation. In addition, the training data set 920 may be used by the software for orienting the organic polarized object 903A as well.
[0017] Figure 9B is a diagrammatic process flow illustrating step A and step B of Figure 7, according to one or more embodiments. In one or more embodiments, a process 950 illustrates the step A of Figure 7 and a process 975 illustrates step B of Figure 7. In one or more embodiments, in the process 950 illustrated in Figure 9B, an image 102 of the ideal organic polarized object 103A may be captured. Further, the image 702 of the ideal organic polarized object 903A may be analyzed to determine the edge and center 914 of the ideal organic polarized object 903A. Furthermore, the vectors 912 may be computed using the image 702 of the ideal organic polarized object 903A. Furthermore, the training coordinates may be recorded as illustrated in step 910 of Figure 9A. In one or more embodiments, the process 975 differs from the process 950 as the process 950 is for obtaining information about the ideal organic polarized object 903A and the process 975 is for obtaining information about the other organic polarized objects for comparing with the training data generated from the ideal organic polarized object 903A for selection/rejection of other organic polarized objects for further procedure.
[0018] In one or more embodiments, in process 975, a raw image of any other organic polarized object, for example, the image of the organic polarized object 907A may be captured and processed. In one or more embodiments, the edges may be determined using the captured image to create an outline image. Further, a binary image 955 that displays the contours of the outline image of 907A of the organic polarized object 907A may be generated using the processed image. In one or more embodiments, the center may be determined using the steps as described. In one or more embodiments, a center of the vector set of a control object may be placed on each pixel in which the single monochrome color is present in the binary image 955. In one or more embodiments, a vote may be casted at an opposite end of each vector of the vector set whose center is placed on each pixel in which the single monochrome color is present in the binary image 955. Furthermore, in one or more embodiments, vote casting may be repeated at the opposite end of each vector of the vector set whose center is placed on each pixel in which the single monochrome color is present in the binary image 955 until the vector set has been scaled “S” number of times and rotated “R” number of times (S and R are positive integers).
[0019] Further, data collected from the voting may be stored until the vector set has been scaled “S” number of times and rotated “R” number of times generating votes at X, Y coordinates of the binary image 955 and saved in vote compare tables created individually for each unique S and R pair.
Further, a particular X and Y coordinate may be located with the highest vote count among any vote compare table. The center of the contours of the binary image 955 may be located at the particular X and Y coordinate if a vote count at the particular X, Y coordinate exceeds a threshold minimum vote count. Further, the size and orientation of the organic polarized object 907A may be determined from the S, R value respectively associated with the vote compare table on which the particular X, Y coordinate is found if the vote count at the particular X,Y coordinate exceeds the threshold minimum vote count.
[0020] In one or more embodiments, after determining the image center of the contour, a physical three dimensional world coordinates of the organic polarized object 107A may be determined as described. In one or more embodiments, a camera model algorithm may be applied that considers a focal length of the image capture device 106, a distortion of the image capture device 706, a warping factor, a distance from a center of a lens of the image capture device 706 to an internal projected image location of the image capture device 706, and a distance between an estimation of the center of the organic polarized object 907A and the center of the lens of the image capture device 706. Furthermore, the estimation of the three dimensional world coordinates of the center of the organic polarized object 907A may be determined by averaging of a distance between a surface under the organic polarized object and the center of the lens, and a closest physical point of the organic polarized object to the center of the lens.
[0021] Figure 10 is a schematic view illustrating generation of a training data set 920, according to one or more embodiments. As described above, in one or more embodiments, an image of the ideal organic polarized object 903A may be captured using the image capture device 1006 and communicated to the data processing system 1008. In one or more embodiments, a data table of vectors 1016 based on the vector information of the ideal organic polarized object 903A may be generated using suitable methods. In one or more embodiments, the generated data table of vectors 1016 may be used to obtain the training data set 1020. The training data set 1020 may be composed of transformations of the data table of vectors 1016 associated with one or more organic polarized objects.
[0022] Figure 11 is a flow chart illustrating a process of comparing a data of an organic polarized object of interest with the training data set 1120, containing the necessary vector information on rotation and scaling of the data table of vectors 1116 to identify if the organic polarized object of interest possesses the desirable shape and /size, as specified in the training data set 1120, according to one or more embodiments. Aforementioned process may be repeated for other organic polarized objects of interest. The images of the organic polarized objects of interest may be captured through the image capture device 106 provided thereof. The images may be processed by the data processing system 1108. In one or more embodiments, in operation 1102, the training data set 1120 may be used for casting votes for determining a center of the organic polarized object of interest. The data table of vectors 1116 may be rotated, scaled and voting process may be performed. The vote counts for each rotation and scaling may be compared to determine the orientation, size, shape, and location of the organic polarized objects. A particular orientation, size, and /or location for which the vote counts are highest may be selected. In one or more embodiments, if the highest vote count generated is above a specified threshold, then the organic polarized object is considered to be identified and is chosen for placement in operation 1104.
[0023] Furthermore, if there is no substantial match in information between the organic polarized object of interest and the information in the training data set 1120 of the ideal organic polarized object 903A, then in operation 1106, it may be determined whether all rotations and scaling of data table of vectors 1116 is performed (e.g., by comparing vote count information obtained at each rotation and scaling of organic polarized object with the data table of vectors 216 of the ideal organic polarized object 903A). Furthermore, if it is determined that all rotations and scales of the data table of vectors 1116 are checked and there is no substantial match between the organic polarized object of interest and the ideal organic polarized object, then in operation 1108, the organic polarized object of interest may be rejected. In one or more embodiments, in operation 1110, rotation and scaling operation may be continued. Further, operation 1102 may be initiated to determine a match and the process is continued until the organic polarized object of interest is matched with the ideal organic polarized object 903A or else the organic object of interest is rejected for not matching.
[0024] In one or more embodiments, the robotic end effector 1610, shown in Figure 16, may be driven by pneumatic cylinders. Furthermore, the pneumatic cylinders may be driven by compressed air. The compressed air may be provided by appropriate means controlled using the software. In alternate embodiments, the robotic end effector 1610 may also be controlled through gear systems, or belt drives, or in any suitable mode of driving. In one or more embodiments, the robotic end effector 1610 may be controlled through a control unit (e.g., memory, controller, and other necessary circuitry) of the robotic end effector 1610. In one or more embodiments, the instructions may be generated by the control unit based on the training data set 1120.
[0025] Figure 16 is a continuation of the process of Figure 11, illustrating additional operations, according to one or more embodiments. In particular, Figure 16 illustrates step C of Figure 7. In one or more embodiments, the selected organic polarized objects may be placed into the tray 712 provided thereof. The subsequent selected organic polarized objects (e.g., 103B, 1611B, 1613B, 1615B, 1619B, 1621B, etc) may be placed continuously thereafter. An alert may be communicated to the data processing system 708 as processing system if the tray 712 becomes full. In addition, a signaling device (e.g., a weight based communication device, count based communication device, inventory device, and the like) may be used to indicate that the tray 712 has reached a maximum capacity. In alternate embodiments, other indicators may be used to indicate to an operator that the tray 712 has reached the maximum capacity. Furthermore, the alerts may be communicated to the data processing system 708 through wired communication or through wireless communication (e.g., Wifi, Wibree, or any suitable module). The signal device may also be configured to prompt/request for a change of tray 712 when the tray is full. In one or more embodiments, a new tray 712 may be placed automatically once the tray 712 is full or based on the request. It should be noted that the tray 712 may be a part of a larger automated assembly system.
[0026] Furthermore, in one or more embodiments, the robotic end effector 1610 may also be configured by the data processing system 708 to pick up the organic polarized object in a specific location and orientation and place the organic polarized object in a specified location and orientation. For example, in case of flower pots, the flower pots may have to be carried in an upright position and placed in a same orientation in the slot provided. In another example, in a case of tulip bulbs, the tulip may be picked up and placed such that the broad end is facing up and narrow end is facing down (e.g., see Figure 16). In alternate embodiments, pins may be provided in the place of slots. The robotic end effector 1610 may be configured to place the tulip bulbs on the pins in a specified orientation.
[0027] In one or more embodiments, Figure 16 particularly illustrates orientation based placing. Figure 16 also illustrates an organic polarized object having a physical shape with a first end 1602 that is narrow in nature and a second end 1600 that is broad in nature. In one or more embodiments, specific type of organic polarized objects may have substantially similar physical structure. Such organic polarized objects may have to be placed in a particular orientation. In addition, the tray 712 or the holder may be specifically designed based on the shape of the organic polarized object. The tray 712 described herein may include one or more slots to hold the organic polarized objects. The slots described herein may be designed based on needs (e.g., broad opening and narrow base to hold plant bulbs).
[0028] In the example embodiment, the shape of the organic polarized object may be analyzed by the data processing system 708. In addition, instructions may be provided by the data processing system 708 to the robotic end effector 1610 to orient the organic polarized object in a particular angle so that the organic polarized object may be picked up and placed in a required orientation. Figure 16 illustrates the robotic end effector 1610 placing an organic polarized object 103B in slots of the tray 712. The slot may be designed in a specific shape such that the organic polarized object 1607B may be received in an orientation in a receptacle 1604 of the slot such that a first end 1602 is received towards the narrow base 1605 of the slot and a second end 1600 is at the top of the slot towards a broad opening 1603 of the slot. The organic polarized object 103B may be placed in the slot of the tray 112 to fit the shape of slot of the tray 712. Alternatively, the tray 1612 may also be designed based on the shape of the organic polarized object 103B. The organic polarized object 103B may be picked up and placed using a first elongated extension 1622 and a second elongated extension 1624 of the robotic end effector 1610. In one or more embodiments, the first elongated extension 1622 and/ or the second elongated extension 1624 may be made of a soft, strong and flexible material.
[0029] Furthermore, the movement of the first elongated extension 1622 and/ or the second elongated extension 1624 may be monitored by sensor 1620 embedded in the first elongated extension 122 and/ or the second elongated extension 1624. Errors, tilts, malfunctioning of the first elongated extension 1622 and/ or the second elongated extension 1624 may be detected by the sensors and may be communicated to the control unit of the robotic end effector 1610 as well as the data processing system 108. Furthermore, necessary corrections may be performed and verified using the sensor 120. In addition, a pressure for lifting the organic polarized object 1607B may be input to the data processing system 1608. Alternatively, the robotic end effector 1610 may be programmed to apply sufficient pressure to lift the organic polarized object 103B. Examples of the sensor 1620 used herein may include, but are not limited to one or more of a pressure sensor, a resistive sensor, a capacitive sensor, and an inductive sensor.
[0030] Figure 17 is a schematic view illustrating a system of organic polarized object detection and placement, according to one or more embodiments. The system includes a polarized object module 1700. The polarized object module 1700 includes, but is not limited to an image module 1702, an image capture module 1704, an organic object detector module 506, a training set module 1708, an algorithm module 1710, a transport module 1712, a quality assurance module 1714, an alert module 1716, a calibration module 1718, a movement module 1720, a change module 1722, and an inventory control module 1724. The image capture device 706 may be configured by the image capture module 1704 to capture the images periodically and to communicate to the captured images to the training set module 17508. The captured image may be processed by the image module 17502 based on an algorithm. In addition, the presence of any organic polarized object at the end effector 1610 may be detected by an organic object detector module 1706. In one or more embodiments, the organic object detector module 1706 may include organic sensors such as cell wall detector 1728, water detector 1732, ATP detector 1726, and/or a carbon detector 1730.
[0031] Furthermore, the dimension data based on the image data of the captured image 702 may be calculated using the algorithm module 1710. The training data set 920, consisting of data tables of vectors, may be generated using the organic polarized object image data by the training set module 508 using the algorithm module 1710. Furthermore, the vote counts of the each of the other organic polarized objects may be generated with the training data set 920 for choosing an object that is substantially similar to the ideal organic polarized object 703A. Based on the result, the instructions may be generated for the robotic end effector 1610 to pick up the organic polarized object (e.g., if the comparison evaluates to be true), or else to reject the organic polarized object (e.g., if the comparison evaluates to be false). The result data may be communicated to the transport module 1712 to generate instructions for the robotic end effector 1610 to perform a specific function. The coordinate system of the robotic end effector 1610 and image capture device 706 may be aligned by the calibration module 1718 to grasp the organic polarized object.
[0032] The robotic end effector 1710 may be directed by the movement module 1720 to perform a specific movement (e.g., alignment, orientation, etc.) based on the dimension data of the organic polarized object of interest. In addition, the robotic end effector 1610 may be configured by the movement module 1720 to perform tasks with available degrees of freedom (DOF). In one or more embodiments, the robotic end effector 1610 may be designed to have ‘n’ DOF (e.g., where ‘n’ is any positive integer) of movement. In one or more embodiments, the robotic end effector 1610 may be designed with ‘n’ DOF for movement that includes, but not limited to, a moving up and down in heaving, a moving left and right in swaying, a moving forward and backward in surging, a tilting forward and backward in pitching, a turning left and right in yawing, a full axis motion with 360 degree rotation, a tilting side to side in rolling, and a moving along one or more of x, y, and z coordinate axes. In one or more embodiments, the movement and functionalities may be controlled through the movement module 1720 controlled using the processor of the data processing system 708. Furthermore, the selected object may be picked up and placed into the tray 712. In one or more embodiments, the process is continued until all the slots in the tray 712 are filled.
[0033] Furthermore, a request for the new tray 712 may be communicated by the change module 1722. In one or more embodiments, an alert may be communicated to by the alert module 1716 to indicate to the data processing system 708 that the tray 712 is full. In one or more embodiments, the change module 1722 may be triggered by the alert module 1716. The alert module 1716 may be controlled by the inventory control module 1724. The quantity to be displaced, the speed of displacement, data processing, modification of algorithms, etc. may be controlled through the inventory control module 1724. In one or more embodiments, the operator may be provided with an interface to configure the settings of the system through the inventory control module 1724. Any odd organic polarized object (e.g., varying significantly in size, shape, condition or material) in the set of organic polarized objects may be rejected or separated through the quality assurance module 1714. The determination of the odd organic polarized object in the set of organic polarized objects may be performed by the quality assurance module 1714 based on a size, shape, material, condition etc. using the detectors provided herein.
[0063] In one or more embodiments, the robotic end effector 1610 may also be trained for performing tasks associated with the organic polarized objects with respect to a particular organic polarized object. For example, the robotic end effector 1610 may be trained to recognize and plant tulip bulbs. The tulip bulbs may be recognized, properly oriented and then planted in a grid provided thereof. In one or more embodiments, the robotic end effector 1610 may be trained specifically through the user interface provided. Furthermore, in one or more embodiments, the robotic end effector 1610 in the automated system may also be trained to simply pick up and place the objects. The embodiments described herein may be used for planting crops (e.g., tulips), arranging the organic polarized objects for packing, choosing the best organic polarized objects among the organic polarized objects.
[0064] Figure 12 illustrates a right side view of the apparatus 1200 for positioning an organic polarized object at the predetermined location with the predetermined orientation, in accordance with one or more embodiments. The apparatus 1200 may include a robotic end effector 1202, one or more pneumatic cylinders, for example, a first pneumatic cylinder 1212, a second pneumatic cylinder 1214, and a third pneumatic cylinder 1216, an image capture device 1220, one or more elongated extensions (e.g. a first elongated extension 1222, a second elongated extension 1224), and a valve (not shown). The robotic end effector 1202 may be trained to capture an image of an organic polarized object through the image capture device 1220 to determine a shape, a size, a first location and/ or a first orientation of the organic polarized object based on the image. In one or more embodiments, one or more feature extraction techniques including, but not limited to a generalized Hough transform may be used to determine the first orientation, size, shape, and /or the first location of the organic polarized object based on the image data. In one or more embodiments the robotic hand 1204 may include a sensor 1218 to sense various parameters associated with the organic polarized object, the parameters including one or more dimensions, a stress withstanding capacity, and one or more contours on a surface of the organic polarized object. Examples of the sensor include, but are not limited to an infra-red device, a biosensor, a color sensor, a heat sensor, and a liquid sensor.
[0065] The coordinate system of the robotic end effector 1202 may be calibrated to match coordinate system of the image capture device 120 through a calibration module. Further the robotic end effector 1202 may receive commands from a processor to pick up and place an orgainic polarized object in a predetermined location and orientation by using comparing a training data set with the organic polarized object. Further, the robotic end effector 1202 may adjust the first location and/ or the first orientation of the organic polarized object to a predetermined location and/ or a predetermined orientation. In some embodiments, the predetermined location and/ or predetermined orientation may be preset in an application. In some other embodiments, the predetermined location and/ or the predetermined orientation are selected by a user. In some embodiments, the robotic end effector 1202 may include a user interface to receive the predetermined orientation and/ or the predetermined location selected by the user. In some embodiments, the robotic end effector 1202 may be operatively coupled with an interface external to the robotic end effector 1202 to receive the predetermined orientation and/ or the predetermined location selected by the user. The robotic hand 1204 and/or the robotic end effector 1202 may secure the organic polarized object. The pneumatic cylinders of the robotic end effector 1202 reduce damage to the organic polarized object by adjusting the grip closure around the organic polarized object when the organic polarized object may be secured through the robotic hand 104 and/or the robotic end effector 1202.
[0066] The first elongated extension 1222 and the second elongated extension 1224 may close simultaneously to secure the organic polarized object. In some embodiments, the first elongated extension 1222 and/or the second elongated extension 1224, includes a cuff that is disc-shaped and/ or concave so as to secure the organic polarized object that is curved in shape and delicate in nature. In some embodiments, the elongated extensions and / or cuff is made of a flexible material to reduce the damage to the organic polarized object. Examples of the flexible material include, but are not limited to rubber and flexible plastic. In some embodiments, the first pneumatic cylinder 1212 and the second pneumatic cylinder 1214 regulate the movement of the first elongated extension 1222 and the second elongated extension 1224 respectively. In some embodiments, the third pneumatic cylinder 1216 regulates the movement of the robotic hand 1204. For purposes of illustration, the detailed description refers to a first pneumatic cylinder, a second pneumatic cylinder, and a third pneumatic cylinder; however the scope of the method, the system, and the apparatus disclosed herein is not limited to the first pneumatic cylinder, the second pneumatic cylinder, and the third pneumatic cylinder but may be extended to include an almost unlimited number of actuators.
[0067] Multiple pneumatic cylinders may be used to regulate the movement of the robotic end effector 1202 and various components of the robotic end effector 1202 therein. In some embodiments, damage to the organic polarized object may be reduced through the valve by adjusting the pressure of a compressed air of one or more of the pneumatic cylinders when the organic polarized object may be secured through the robotic hand 1204. In some embodiments, the valve also regulates and/ or synchronizes closing of the first elongated extension 1222 and the second elongated extension 1224 such that the first elongated extension 1222 and the second elongated extension 1224 close simultaneously and in a regulated manner. In one or more embodiments, one or more motions of the robotic end effector 1202 includes “n” degrees of freedom of movement. The “n” degrees of freedom of movement includes, but is not limited to, a moving up and down in heaving, a moving left and right in swaying, a moving forward and backward in surging, a tilting forward and backward in pitching, a turning left and right in yawing, a full axis motion in 360 degree rotation, a tilting side to side in rolling, and a moving along one or more of x, y, and z coordinate axes.
[0068] The system disclosed herein may include the image capture device 1220 to capture the image of the organic polarized object. Examples of the image capture device 1220 may include, but is not limited to, a digital camera/cameras, a video camera, a probe, an optical device, an infra-red device, a biosensor, a color sensor, a heat sensor, a water sensor and a laser device. The captured image of the organic polarized object may be converted into an image data of the organic polarized object. The image data may be later used to determine the first location, first size and/ or the first orientation of the organic polarized object. Examples of the image data includes but is not limited to, one or more dimensions of the organic polarized object, a three-dimensional structure of the organic polarized object, mass and density of the organic polarized object and/ or a shape of the organic polarized object. The system also includes the robotic end effector 1202 to adjust the first location and/ or the first orientation of the organic polarized object to the predetermined location and/ or the predetermined orientation respectively. In one or more embodiments, the system further includes a processor operatively coupled to the robotic end effector 1202 to determine the first location and/ or the first orientation of the organic polarized object. In one or more embodiments, the processor automates the positioning of the organic polarized object to the predetermined location with a predetermined orientation.
[0069] In one or more embodiments, the predetermined location and/ or the predetermined orientation are selected by a user. In these embodiments, the system includes a user interface to receive the predetermined location and/ or the predetermined orientation from the user. The system furthermore includes a robotic hand 104 operatively coupled to the robotic end effector 102 to secure the organic polarized object without damaging the organic polarized object. The elongated extensions of the robotic hand 104 may be made of a flexible material to reduce the damage to the organic polarized object. In one or more embodiments, the system furthermore includes the valve configured to reduce damage to the organic polarized object on securing the organic polarized object through the robotic end effector 102. In one or more embodiments, the system includes a tray including slots or pins to position one or more organic polarized objects in the slots/ pins of the tray through the robotic end effector 1202. During positioning the organic polarized objects in the tray, a first end of the organic polarized object may be oriented towards a narrow base of the slot/pin and a second end may be oriented towards a broad opening of the slot/pin. The organic polarized objects may be positioned in the slots/pins of the tray to encourage growth of the organic polarized objects in the tray. In one or more embodiments, a hydroponic technology may be used to grow the organic polarized objects. As used herein the term hydroponics refers to a method of growing plants without soil. In these embodiments one or more essential nutrients may be introduced to the organic polarized objects through fluids (e.g., water) in the place of soil. In some embodiments, the system further includes a conveyor belt to transport the organic polarized object from storage of the organic polarized object to the robotic end effector 1202.
[0070] Figures 13-15 illustrate positioning an organic polarized object 1302 at a predetermined location with a predetermined orientation 1510, in accordance with one or more embodiment. A robotic end effector 1302 determines a first orientation 1304 of the organic polarized object 1302. Figure 14 exemplarily illustrates securing the organic polarized object 103 between the first elongated extension 1322 and the second elongated extension 1324 of the robotic end effector 1402. During securing the organic polarized object 103, the robotic end effector 1402 senses various parameters associated with the organic polarized object 103, the parameters including one or more dimensions, a stress withstanding capacity, and one or more contours on a surface of the organic polarized object 103, using a sensor 1418. Examples of the sensor include, but are not limited to an infra-red device, a biosensor, a color sensor, a heat sensor, and a liquid sensor. Based on the sensed parameters, the robotic end effector 1402 may compute a required pressure to secure the organic polarized object 103 without causing damage or deformation to the organic polarized object 103. In one or more embodiments the robotic end effector 1402 determines the various parameters associated with the organic polarized objects through the image capture device 1420. The image capture device 1420 includes, but is not limited to, a digital camera, a video camera, a probe, an optical device, an infra-red device, a biosensor, a color sensor, a heat sensor, a water sensor, and a laser device.
[0071] The robotic end effector 1402 then secures the organic polarized object 103 by exerting the required pressure through the first elongated extension 1322 and the second elongated extension 1324. Also, a first pneumatic cylinder 1412 regulates the movement of the first elongated extension 1322 and a second pneumatic cylinder 1314 regulates the movement of the second elongated extension 1324 during securing the organic polarized object 103. Further, a third pneumatic cylinder 1416 regulates movement of the robotic hand 1404. The robotic end effector 1402 then senses one or more coordinates of the predetermined location, for example, the robotic end effector 1402 may sense a slot 1508 in a tray 1506 to position the organic polarized object 103 in the slot 1508 and proceeds towards the slot 1508. Robotic end effector 1502 adjusts the organic polarized object 103 from first orientation 1504 to predetermined orientation 1510. The predetermined orientation 1510 may be determined by the robotic end effector 1502 based on a structure of the slot 1508 of the tray 1506. A first end, for example, a proximal end of the organic polarized object 1502 may be oriented towards a narrow base of the slot 1508 and a second end, for example, a distal end may be oriented towards a broad opening of the slot 1508. The robotic end effector 1502 then positions the organic polarized object 103 in the slot 1508 by releasing the organic polarized object 103 secured between the first elongated extension 1322 and the second elongated extension 1324 as exemplarily illustrated in Figure 15.
[0072] Figure 18 is a block diagram of a processing system 1800 of organic polarized object detection and/ or positioning, in accordance with one or more embodiments. The processing system 1800 includes a processor 1802 operatively coupled with a bus 1804. The processor 1802 controls and processes various functionalities of the processing system 1800. The processor 1802 may include or may be operatively coupled to one or more of a calibration module 1803, an image module 1806, an image capture module 1808, an organic object detector module 1810, a training set module 1820, an algorithm module 1822, a transport module 1824, a quality assurance module 1826, an alert module 1828, a movement module 1830, a change module 1832, an inventory control module 1834, and a basket flipping module 1835.
[0073] The calibration module 1803 may align the coordinates systems of the robotic end effector 1502 with coordinate system of the image capture device 1420. The image capture module 1808 may configure an image capture device 120 to capture the images of the organic polarized objects periodically and communicate the images to the training set module 1820. The captured image may be processed by the image module 1806 based on an algorithm. In addition, the presence of any organic polarized object may be detected by an organic object detector module 1810. In one or more embodiments, the organic object detector module 1810 may include organic sensors such as cell wall detector 1812, a water detector 1814, ATP detector 1816, and a carbon detector 1818. Furthermore, the dimension data based on the image data of the captured image may be calculated using the algorithm module 1822. The training data set may be generated using the dimension data of the organic polarized object by the training set module 1820 using the algorithm module. In one or more embodiments, the organic object detector module 1810 may coordinate with the image capture module 1808. The images may be captured by the image capture module 1808 as and when the organic polarized objects are detected by the organic object detector module 1810.
[0074] Similarly, the dimension data of other organic polarized objects may be generated. Furthermore, the dimension data of the each of the other organic polarized objects may be compared with the training data set for choosing an object that is substantially similar to an ideal organic polarized object. Based on the result, the instructions may be generated for the robotic end effector 1502 to grasp the organic polarized object, for example, if the comparison evaluates to be true, and to reject the organic polarized object for example, if the comparison evaluates to be false. The result data may be communicated to the transport module 1824 to generate instructions for the robotic end effector 1502 to perform a specific function.
[0075] The robotic end effector 1502 may be directed by the movement module 1830 to perform a specific movement, for example, pick up and place the organic polarized object based on the dimension data of the organic polarized object. In addition, the robotic end effector 1502 may be configured by the movement module 1830 to perform tasks with available degrees of freedom. In one or more embodiments, the robotic end effector 1502 motions may be designed to have ‘n’ degrees of freedom of movement. In one or more embodiments, the robotic end effector 1502 may be designed with degrees of freedom of movement that includes, but not limited to, a moving up and down in heaving, a moving left and right in swaying, a moving forward and backward in surging, a tilting forward and backward in pitching, a turning left and right in yawing, a full axis motion with 360 degree rotation, a tilting side to side in rolling, and a moving along one or more of x, y, and z coordinate axes. Furthermore, the selected object may be picked and placed into the tray 712. In one or more embodiments, the process may be continued until all the slots in the tray 712 are filled.
[0076] Furthermore, a request for the new tray may be communicated by the change module 1832. In one or more embodiments, an alert may be communicated by the alert module 1828 to indicate the processing system 1800 that the tray 712 may be full. In one or more embodiments, the change module 1832 may be triggered by the alert module 1828. The alert module 1828 may be controlled by the inventory control module 1834. The quantity to be displaced, the speed of displacement, data processing, modification of algorithms, and the like may be controlled through the inventory control module 1834. In one or more embodiments, the operator may be provided with an interface to configure the settings of the system through the inventory control module 1834. Any odd organic polarized object for example, varying significantly in, but not limited to, color, shape, size condition or material, in the set of organic polarized objects may be rejected or separated through the quality assurance module 1826. The determination of the odd organic polarized object in the set of organic polarized objects may be performed by the quality assurance module 1826 based on, but not limited to, a size, color, shape, material, and the like using the detectors provided herein. The basket flipping module 1835 may communicate with the transport module 1824 to coordinate flipping the basket over. In one or more embodiments, the basket flipping module 1835 may also be configured to control and/or coordinate removal of the tray after flipping the basket over.
[0077] Further, the processing system 1800 also includes a memory 1836 such as a random access memory (RAM) or other dynamic storage device, coupled to the bus 1804 for storing information which can be used by the processor 1802. The memory 1836 can be used for storing any temporary information required, for example, the dimension data of the organic polarized object, a comparison data of the dimension data and the training data set. The processing system 1800 further includes a read only memory (ROM) 1838 or other static storage device coupled to the bus 1804 for storing static information for the processor 1802. The processing system 1800 can be coupled via the bus 1804 to a display unit 1840, such as a cathode ray tube (CRT), a liquid crystal display (LCD) or a light emitting diode (LED) display, for rendering the display images to one or more users. An input device 1842 including alphanumeric and other devices, may be coupled to the bus 1804 for communicating an input to the processor 1802. The input device 1842 can be included in the processing system 1800.
[0078] Another type of input device 1842 may be a cursor control, such as a mouse, a trackball, or cursor direction keys for communicating the input to the processor 1802 and for controlling cursor movement on the display unit 1840. The input device 1842 can also be included in the display unit 1840, for example a touch screen. In some embodiments the processing system 1800 may coupled via the bus 1804 to a user interface 1844. In one or more embodiments, the robotic end effector 1502 may also be trained for performing tasks associated with the organic polarized objects with respect to a particular organic polarized object. For example, the robotic end effector 1502 may be trained to recognize and plant tulips/ tulip bulbs. The tulip bulbs may be recognized, properly oriented and then planted in a grid provided thereof. In one or more embodiments, the robotic end effector 1502 may be trained specifically through the user interface 1844 provided. Furthermore, in one or more embodiments, the robotic end effector 1502 in the automated system may also be trained to simply pick up and place the objects. The embodiments described herein may be used for planting crops, for example, tulip bulbs, arranging the organic polarized objects for packing, and choosing the best organic polarized objects among the organic polarized objects.
[0079] Figure 19A is a process flow illustrating identification of organic polarized objects and placement of the same using the automated system, according to one or more embodiments. In operation 1902, an image of a first organic polarized object (e.g., an ideal organic polarized object) may be captured using the image capture device. In operation 1904, a first image data of the first organic polarized object (e.g., ideal organic polarized object) may be collected. In one or more embodiments, the image data may be collected by the data processing system. In operation 1906, a first dimension data of the first organic polarized object including, but not limited to a center and edges of the first organic polarized object may be algorithmically calculated using the processor of a data processing system (e.g. data processing system). In operation 1908, a first data table (e.g., vector data) of the dimension data of the first organic polarized object 103A may be created. In operation 1910, the training data set may be formed by rotating and scaling and / or transforming (e.g., skewing, rotating) the first data table of the first organic polarized object 103A. The aforementioned operations are associated with the steps A of Figure 5 and/or Figure 7. In operation 1912, an image of a second organic polarized object may be captured using the image capture device. In operation 1914, the captured image of the second organic polarized object may be collected (e.g., using the data processing system). In operation 1916, high vote count data may be calculated for the second image data of the captured image of the second organic polarized object using the training data set.
[0080] Figure 19B is a continuation of process flow of Figure 19A, illustrating additional operations, according to one or more embodiments. In operation 1918, a second dimension data of the second organic polarized object including, but not limited to the center and edges of the second organic polarized object may be algorithmically calculated using the processor. In operation 1920, the second dimension data may be determined as a distinct data for the second organic polarized object even if the second organic polarized object is in a position that may be adjacent, a bordering, a overlapping, an underneath to the other object and an up-side down state. In operation 1922, the training data set may be compared with the second dimension data of the second organic polarized object to identify the second organic polarized object and precise size, shape, orientation and location data of the second organic polarized object using the processor.
[0081] In operation 1924, the robotic end effector movement having an “n” degree of freedom of movement may be configured to pick up the second organic polarized object. The “n” degrees of freedom of movement is one or more of a moving up and down in heaving, a moving left and right in swaying, a moving forward and backward in surging, a tilting forward and backward in pitching, a turning left and right in yawing, a full axis motion in 360 degree rotation, a tilting side to side in rolling, and moving along one or more of x, y, and z coordinate axes. The operations from 1912 to 1924 may represent the step B of Figure 5 and/or Figure 7.
[0082] In operation 1926, the second organic polarized object may be picked up using the robotic end effector and the precise location/orientation data. In one or more embodiments, the second organic polarized object may be picked up only if the comparison evaluates to be true. In operation 1928, the second organic polarized object may be transported from a first location and first orientation to a second location and second orientation in a specific tray with two or more slots.
[0083] Figure 19C is a continuation of process flow of Figure 19B, illustrating additional operations, according to one embodiment. In operation 1930, the slot in a specific shape may be used to receive the first end of the second organic polarized object into the receptacle of the slot before the second end of the second organic polarized object, such that the first end of the second organic polarized object is oriented towards a narrow base of the slot and the second end is oriented towards a broad opening of the slot. In operation 1932, an angle of deposit may be corrected. The correction may be based on the type of the slot. In operation 1934, the second organic polarized object may be deposited in the precise orientation to the slot in the tray (e.g., as illustrated in Figure 16). The operations from 1926 to 1934 may represent the step C of Figure 5 and/or Figure 7. In operation 1936, once the tray is full, a new tray would be placed and the process would start again from step A, followed by step B and then step C till the tray is filled. Operation 1936 may represent step D of Figure 5 and/or Figure 7. Once the tray is full step D continues.
[0084] Figure 20 is a process flow illustrating generating an orientation, size and the three-dimensional coordinates of the organic object of interest and communication of the same to the robotic end effector. In one or more embodiments, in operation 2002, a raw image of the organic polarized object may be generated from the image capture device. In one or more embodiments, in operation 2004, edges of the organic polarized object may be determined from the raw image to create an outline image. In one or more embodiments, appropriate edge detection algorithms or tools may be used for detecting the three-dimensional shape of the organic polarized object. In operation 2006, a binary image that displays the contours of the outline image in a single monochrome color may be created. In operation 2008, a center of the contours of the binary may be determined. In operation 2010, physical three dimensional coordinates of the organic polarized object may be determined. In operation 2012, the orientation, size, and the physical three dimensional coordinates of the organic polarized object may be communicated to the robotic end effector.
[0085] Figure 21 is a process flow diagram of determining a center of the contour of the binary image, according to one or more embodiments. In operation 2102, a center of the vector set of a control object may be placed on each pixel in which the single monochrome color is present in the binary image. In operation 2104, a vote at an opposite end of each vector of the vector set may be casted whose center is placed on each pixel in which the single monochrome color is present in the binary image. In operation 2106, a casting of the vote may be repeated at the opposite end of each vector of the vector set whose center is placed on each pixel in which the single monochrome color is present in the binary image until the vector set has been scaled “S” number of times and rotated “R” number of times. In operation 2108, data collected from the voting may be stored until the vector set has been scaled “S” number of times and rotated “R” number of times generating vote counts at X, Y coordinates of the binary image in vote compare tables created individually for each unique S, R pair. In operation 2110, a particular X, Y coordinate may be located with the highest vote count among any vote compare table. In operation 2112, center of the contours of the Binary Image may be located at the particular X,Y coordinate if a vote count at the particular X, Y coordinate exceeds a threshold minimum vote count.
[0086] Figure 22 is a process flow illustrating a method of automated organic polarized object organization, in accordance with one or more embodiments.
As used herein, the term “organic polarized object” refers to any organic object of a regular or an irregular shape and form, having a proximal end and a distal end, and hence an orientation. Examples of the organic polarized object include, but are not limited to, a seed, a plant bulb, a resting stage of a seed plant, and a sapling. The organic polarized object may be positioned through a robotic end effector. As used herein, the term organization refers to positioning one or more organic polarized objects in a predetermined orientation and /or at a predetermined location.
[0087] In one or more embodiments, in operation 2202, an organic polarized object may be spread in a single file on the conveyor belt when the organic polarized object may be transported from an organic polarized object storage. In one or more embodiments, in operation 2204, an inventory control system may be updated through a wireless communication system when a count of the organic polarized object may be communicated. Examples of the wireless communication system includes, but is not limited to a Bluetooth, a Zigbee, a WiFi, a WiMax, a power over ethernet (POE), and a Wibree.
[0088] In one or more embodiments, in operation 2206, the organic polarized object may be automatically filled in a slot of a tray or affixed to pins in the tray through a robotic end effector until a maximum fill capacity of the tray may be achieved. The tray may be made of one or more of a biodegradable material, a plastic material, and a reusable material. The slot may be of a specific shape to receive a first end of the organic polarized object in the slot before a second end of the organic polarized object, such that the first end of the organic polarized object may be oriented towards a narrow base of the slot and the second end may be oriented towards a broad opening of the slot.
[0089] In one or more embodiments, in operation 2208, the tray may be covered with a nutrient mixture and later with a basket. In one or more embodiments, personnel may be updated about the status of an inventory item. The inventory item includes, but is not limited to the organic polarized object, the nutrient mixture, the tray and the basket. Updating personnel may be conducted via a wired/ wireless communication, including, for example, through a cell phone, a PDA and /or a computer.
[0090] In one or more embodiments, in operation 2210, the basket comprising the tray filled with the organic polarized object and the nutrient mixture may be inverted / flipped so that the organic polarized objects are in the correct orientation for growing. In one or more embodiments, in operation 2212, the tray may be removed. In one or more embodiments, in operation 2213, the organic polarized objects may be covered with the nutrient mixture while in the basket.
[0091] In one or more embodiments, the basket containing the organic polarized objects with the nutrient mixture may be stacked in a vertical form and /or a horizontal form on a platform in a cold storage facility for a length of time. In one or more embodiments, the stacked basket may be transported to a growing area. In one or more embodiments, in operation 2214, the basket and /or the basket stack may be stored in the cold storage facility until a planting season time arrives.
[0092] In one or more embodiments, the organic polarized objects not selected by the robotic end effector for placement in the slot of the tray may be recycled. In one or more embodiments, a hydroponic technology may be used to grow the organic polarized objects. As used herein the term hydroponics refers to a method of growing plants without soil. In these embodiments one or more essential nutrients may be introduced to the organic polarized objects through fluids (e.g., water) in the place of soil. Also, in these embodiments, the tray may contain an array of sharp pins and the robotic end effector may stick the organic polarized objects to the sharp pins. The trays may then be positioned in a liquid (e.g., water) to allow the organic polarized objects to grow when submerged in the liquid.
[0093] Figures 23A-23B show a process flow illustrating a vision based method of automated organic polarized object organization, in accordance with one or more embodiments. In one or more embodiments a robotic end effector may be trained to automatically organize one or more organic polarized objects using a training data set. In order to form the training data set, in one or more embodiments, in operation 2302, a first image of a first organic polarized object may be captured using an image capture device. The image capture device may include, but is not limited to, a digital camera, a video camera, a probe, an optical device, an infra-red device, a biosensor, a color sensor, a heat sensor, a water sensor, and a laser device. In one or more embodiments, in operation 2304, a first image data of the first organic polarized object may be collected from the first image.
[0094] In one or more embodiments, in operation 2306, the first dimension data of the first organic polarized object may be algorithmically calculated using a processor. The first dimension data includes, but is not limited to, one or more of a width, a depth, a length, a distance, an intensity, a curvature, a surface area, a volume, a narrow field, a broad field, center, edges and an angle. In one or more embodiments, in operation 2308, first data table of the first dimension data of the first organic polarized object may be created. In one or more embodiments, in operation 2310, a training data set may be formed by transforming (e.g., skewing, rotating) the first data table of the first organic polarized object.
[0095] In one or more embodiments, in operation 2312, a second image of a second organic polarized object may be captured using the image capture device. In one or more embodiments, in operation 2314, a second image data of the second organic polarized object may be collected using the captured second image. In one or more embodiments, in operation 2316, a second dimension data of the second organic polarized object may be algorithmically calculated using the second image data through a processor. The second dimension data includes, but is not limited to, one or more of a width, a depth, a length, a distance, an intensity, a curvature, a surface area, a volume, a narrow field, a broad field, edges, center and an angle. In one or more embodiments, in operation 2318, a high vote count data may be calculated based on the second dimension data and the training data.
[0096] In one or more embodiments, in operation 2320, a training data set may be compared to the second dimension data to identify the second organic polarized object and /or a precise location, orientation, size, and /or shape data of the second organic polarized object using a processor. In some embodiments, one or more feature extraction techniques including, but not limited to a generalized Hough transform may be used for comparison and/or identification.
[0097] In one or more embodiments, in operation 2322, a dimension data may be determined as a distinct data of the second organic polarized object even if the second organic polarized object may be in one or more of an adjacent, a bordering, overlapping, underneath the first organic polarized object and in an up-side down state. In one or more embodiments, in operation 2324, a robotic end effector movement having “n” degrees of freedom of movement may be selected. The “n” degrees of freedom of movement includes, but is not limited to, a moving up and down in heaving, a moving left and right in swaying, a moving forward and backward in surging, a tilting forward and backward in pitching, a turning left and right in yawing, a full axis motion in 360 degree rotation, a tilting side to side in rolling, and a movement along one or more of x, y, and z coordinate axes. In one or more embodiments, in operation 2326, the second organic polarized object may be picked up using the robot end effector using the precise location, size, and /or orientation data. In one or more embodiments, in operation 2328, a second organic polarized object may be transported from a first place and a first orientation to a second place and a second orientation in a specific tray with a slot.
[0098] Figure 24 is a process flow diagram illustrating a method of positioning an organic polarized object at a predetermined location with a predetermined orientation, in accordance with one or more embodiments. As used herein, the term “organic polarized object” may refer to any organic polarized object of a regular or an irregular shape and form, having a proximal end and a distal end, and hence an orientation. Examples of the organic polarized object may include, but are not limited to, a seed, a plant bulb, a resting stage of a seed plant, and a sapling. The organic polarized object may be positioned through a robotic end effector
[0099] In one or more embodiments, in operation 2401, one or more images of a typical organic polarized object may be captured and a training data set may be created based on the captured images. In one or more embodiments, in operation 2402, an image of a first organic polarized object may be captured through an image capture device. Examples of the image capture device include, but are not limited to, a digital camera, video camera, a probe, an optical device, infra-red device, a biosensor, a color sensor, a heat sensor, a water sensor and a laser device. In some embodiments, the image capture device may be operatively coupled with the robotic end effector. In some other embodiments, the image capture device may be embedded within the robotic end effector. In one or more embodiments, in operation 2404, the captured image of the first organic polarized object may be converted to an image data of the first organic polarized object. In operation 2405, the image data may be converted, through a processor, into a dimension data of the first organic polarized object. Examples of the dimension data may include, but are not limited to, one or more dimensions of the first organic polarized object, a structure of the first organic polarized object, mass and density of the first organic polarized object and a shape of the first organic polarized object, the center of the first organized polarized object and/or the edges of the first organic polarized object.
[00100] In one or more embodiments, in operation 2406, a first location and a first orientation of the first organic polarized object may be determined through a processor, based on the dimension data of the first organic polarized object. In some embodiments, one or more feature extraction techniques, including, but not limited to a generalized Hough transform may be used to determine the first location, orientation, and size of the first organic polarized object based on a comparison of the training data set with the dimension data.
[00101] In one or more embodiments, in operation 2408, a movement of the robotic end effector towards the first organic polarized object, along one or more of an x, a y, and a z coordinate axes, may be guided through the processor. The robotic end effector may have “n” degrees of freedom of movement. The “n” degrees of freedom includes one or more of a moving up and down in heaving, a moving left and right in swaying, a moving forward and backward in surging, a tilting forward and backward in pitching, a turning left and right in yawing, a full axis motion in 360 degree rotation, a tilting side to side in rolling, and a moving along one or more of x, y and z coordinate axes. The robotic end effector may move towards the determined first location and first orientation of the first organic polarized object and thus towards the first organic polarized object using the “n” degrees of freedom of movement based on the guidance. Further, a first pneumatic cylinder of the robotic end effector may regulate and control movement of a first elongated extension of the robotic end effector and/ or a second pneumatic cylinder of the robotic end effector may regulate and control movement of a second elongated extension of the robotic end effector.
[00102] In one or more embodiments, in operation 2410, a pressure may be applied through a robotic end effector to secure the first organic polarized object at the first location with the first orientation. A required amount of pressure to be exerted on the first organic polarized object to secure the first organic polarized object without damaging may be computed through the processor. In some embodiments a stress withstanding capacity of the first organic polarized object may be computed through the processor, using the image data, to determine the required amount of pressure to be exerted on the first organic polarized object to secure without damaging, the first organic polarized object. In some embodiments, damage to the first organic polarized object may be reduced by adjusting the pressure exerted on the first organic polarized object when the first organic polarized object may be secured through the robotic end effector. In some embodiments, the pressure may be adjusted through one or more pneumatic cylinders including the first pneumatic cylinder and/or the second pneumatic cylinder. In some embodiments, one or more portions of the robotic hand may be made of a flexible material to reduce the damage to the first organic polarized object.
[00103] In one or more embodiments, in operation 2412, the first organic polarized object may be moved to a predetermined location and /or a predetermined orientation. The first orientation of the first organic polarized object may be adjusted to the predetermined orientation through the elongated extensions and/or the robotic end effector. The first organic polarized object may be secured between a first elongated extension and a second elongated extension of the robotic end effector to move the first organic polarized object. The movement of the first elongated extension and the second elongated extension may be regulated and synchronized through the pneumatic cylinders. The first pneumatic cylinder and the second pneumatic cylinder from among the pneumatic cylinders may regulate and synchronize the movement of the first elongated extension and the second elongated extension respectively.
[00104] In one or more embodiments, in operation 2414, the first organic polarized object may be positioned at the predetermined location and a predetermined orientation. In one or more embodiments, in operation 2416, the positioning of the first organic polarized object and/or other organic polarized objects at the predetermined location and predetermined orientation may be automated through a training data set. The predetermined location and the predetermined orientation may be automatically chosen or selected by a user. In some embodiments, the positioning of a second organic polarized object of a second size and at a second location may be automated through a comparison with the training data set. The second size may be different from a first size of the first organic polarized object and the second location may be different than the first location, and the second orientation may be different than the first orientation. In some embodiments, the robotic end effector may be programmed to position the organic polarized objects on a conveyor belt and the organic polarized objects may be transferred to the predetermined location through the conveyor belt.
[00105] Although the present embodiments have been described with reference to specific example embodiments, it will be evident that various modifications and changes may be made to these embodiments without departing from the broader spirit and scope of the various embodiments. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. The foregoing examples have been provided merely for the purpose of explanation and are in no way to be construed as limiting of the present invention disclosed herein. While the invention has been described with reference to various embodiments, it is understood that the words, which have been used herein, are words of description and illustration, rather than words of limitation. Further, although the invention has been described herein with reference to particular means, materials and embodiments, the invention is not intended to be limited to the particulars disclosed herein; rather, the invention extends to all functionally equivalent structures, methods and uses, such as are within the scope of the appended claims. Those skilled in the art, having the benefit of the teachings of this specification, may affect numerous modifications thereto and changes may be made without departing from the scope and spirit of the invention in its aspects.

权利要求:
Claims (10)
[1]
A method, comprising: capturing an image of a first organically polarized object using a camera; collecting first image data from the first organically polarized object; algorithmically calculating first dimension data of the first organically polarized object including a center and edges of the first organically polarized object by means of a processor; generating a first data table from the first dimension data of the first organically polarized object; and forming a training data set by at least one transformation and scaling of the first data table of the first organically polarized object.
[2]
The method of claim 1 further comprising: capturing a second organically polarized object using the camera; collecting second image data from the second organically polarized object; algorithmically calculating second dimension data of the second organically polarized object including a center and edges of the second organically polarized object by the processor; generating a second data table from the second dimension data from the second organically polarized object; and calculating a high number of voice data using the second dimension data of the captured image of the second organically polarized object; creating a second data table from the second dimension data from the second organically polarized object; determining the second dimension data as separate data for the second organically polarized object even if the second organically polarized object is in at least one of an adjacent, adjacent, overlapping, below the other object and an upside-down position; comparing the training data set and the second dimension by generating a high number of voice data to at least one of the second organically polarized object and a precise size, shape, orientation and location data of the second organically polarized object using of the processor; selecting a robot effector movement that has n degrees of freedom of movement; picking up the second organically polarized object by means of the robot effector in at least one of a precise location and a precise orientation; transporting the second organically polarized object from a first location and first orientation to a second location and second orientation in a specific carrier with a slot and / or pins; and having the slot in a specific shape to receive a first end of the second organically polarized object in the slot for a second end of the second polarized object, so that the first end of the second organically polarized object is oriented toward a narrow base of the slot and the second end is oriented to a wide opening of the slot.
[3]
The method of claim 1, wherein the size data includes at least one of a width, a depth, a length, a distance, an intensity, a bend, a surface, a volume, a narrow field, a wide field, edges, center and a corner.
[4]
The method of claim 1 further comprising: finding the edges of the first organically polarized object of the captured image to create the training set of data.
[5]
The method of claim 1 further comprising: correcting an angle of placement based on the type of slot.
[6]
The method of claim 5 further comprising: placing the organically polarized object in the precise location and orientation in a slot in the carrier or on a pin.
[7]
The method of claim 1, further comprising: allowing the "n" degree freedom of movement is at least one of an up and down movement in swell, a movement to the left and right in swing, a movement forward and backward in a undulating movement, a tilt forwards and backwards in a slope, a left and right rotation in a yaw movement, a full axis movement in a 360 degree rotation, a left and right rotation in a rolling movement and a movement along at least one of x, y and z coordinate axes.
[8]
The method of claim 1, further comprising: spreading an organically polarized object in one row one after the other on the conveyor when the organically polarized object is transported from an organically polarized object storage space; updating an inventory control system by a wireless communication system when a number of the organically polarized object is communicated. automatically filling the organically polarized object into a slot of a carrier by means of a robot effector until a maximum filling capacity of the carrier is reached, wherein the slot has a specific shape about a first end of the organically polarized object in the slot for a second end of the organically polarized object, so that the first end of the second organically polarized object is oriented to a narrow base of the slot and the second end is oriented to a wide opening of the slot. partially covering the carrier with nutrient mix and a basket, turning the carrier / basket around to orient the organically polarized objects to grow and possibly removing the carrier and adding more nutrient mix to the organically polarized objects in the basket.
[9]
The method of claim 8, further comprising: inverting the basket with the carrier filled with the organically polarized objects and nutrient mix; moving the basket to a nutrient filling station; filling the basket with the carrier with the organically polarized objects with the nutrient mix; and storing the basket in a cold storage room until a planting season arrives.
[10]
The method of claim 1, further comprising: capturing an image of an organically polarized object by an image capturing device; converting, by means of a processor, the captured image of the organically polarized object into image data of the organically polarized object; converting, by the processor, the image data into dimension data for the organically polarized object; determining, by the processor, a first location and a first orientation of the organically polarized object based on a comparison of the dimension data of the organically polarized object and a training data set; guiding, by the processor, a movement of a robot effector to the organically polarized object, along at least one of x, y and z coordinate axes, wherein the robot effector has n degrees of freedom of movement; applying pressure by the robot effector to secure the organically polarized object at the first location with the first orientation; moving the fixed organically polarized object to a predetermined location and a predetermined orientation; positioning the fixed organically polarized object of the predetermined orientation at the predetermined location; and automating, by means of a training data set, the positioning of the organically polarized object to the predetermined location and the predetermined orientation, wherein the predetermined location and the predetermined orientation are at least one of automated or selected by a user may be.

类似技术:

---

公开号 | 公开日 | 专利标题

---

US8504204B2|2013-08-06|Automated positioning of an organic polarized object

---

US8363905B2|2013-01-29|Automated image analysis of an organic polarized object

---

US20120017507A1|2012-01-26|Automated organic polarized object organization

---

US10225993B2|2019-03-12|Method for automating transfer of plants within an agricultural facility

---

EP2086308B1|2011-01-19|Autonomous fruit picking machine

---

Han et al.2012|Strawberry harvesting robot for bench-type cultivation

---

US10870204B2|2020-12-22|Robotic system control method and controller

---

CN101180928A|2008-05-21|Seedling replanting system based on machine vision

---

Eizicovits et al.2016|Integration of perception capabilities in gripper design using graspability maps

---

US20190308822A1|2019-10-10|Apparatus and Method for Placing Bulbs

---

Hayashi et al.2011|Gentle handling of strawberries using a suction device

---

US20180007842A1|2018-01-11|Apparatus and method for planting plant cuttings

---

NL2007152C2|2014-04-07|System and method of polarization of an organic object with automated image analysis and organization.

---

CN104658015A|2015-05-27|Visual locating method for automatic laser slicing in corn breeding

---

NL2022385B1|2019-12-20|Apparatus and Method for Placing Bulbs

---

Yamamoto et al.2012|Study on an automatic packing system for strawberries

---

JP2008190912A|2008-08-21|Image recognition method and gripping device

---

Yung et al.2019|Partially structured robotic picking for automation of tomato transplantation

---

Peng et al.2021|Research progress of urban dual-arm humanoid grape harvesting robot

---

US20200094401A1|2020-03-26|System and method for automatic learning of product manipulation

---

Chiu et al.2012|Integrated test of an autonomous tomato picking robot

---

Pham et al.2020|A Computer Vision Based Robotic Harvesting System for Lettuce

---

JP2021003800A|2021-01-14|Robot system control method, non-transient computer-readable recording medium, and robot system control device

---

JP2008188691A|2008-08-21|Delivery device and delivery system

---

JP2006042686A|2006-02-16|Automatic plant-separating and transplanting system

同族专利:

---

公开号 | 公开日

---

NL2007152C2|2014-04-07|

引用文献:

---

公开号 | 申请日 | 公开日 | 申请人 | 专利标题

---

US4909376A|1987-10-06|1990-03-20|Western Technologies Automation, Inc.|Robotically controlled component feed mechanism visually monitoring part orientation|

---

WO2003022034A1|2001-09-07|2003-03-20|Gartneriet Pkm Aps|Pick and place robot|

---

WO2003103419A1|2002-05-28|2003-12-18|Van Der Knijff Gerard Cornelis|Device for treating bulbous or tuberous plants|

---

US20050114918A1|2003-11-25|2005-05-26|Edwin Hirahara|System and method of embryo delivery for manufactured seeds|

---

法律状态:
2017-03-01| MM| Lapsed because of non-payment of the annual fee|Effective date: 20160801 |

优先权:

---

申请号 | 申请日 | 专利标题

---

US84031510|2010-07-21|

---

US12/840,315|US20120017507A1|2010-07-21|2010-07-21|Automated organic polarized object organization|

---

US12/840,334|US8363905B2|2010-07-21|2010-07-21|Automated image analysis of an organic polarized object|

---

US84033410|2010-07-21|

---

US12/841,191|US8504204B2|2010-07-22|2010-07-22|Automated positioning of an organic polarized object|

---

US84119110|2010-07-22|

---